1. Field
Embodiments of the invention relate to implementing storage management functions using a data store system.
2. Description of the Related Art
A storage subsystem may be described as including one or more host computers and one or more storage servers. The storage servers provide the host computers access to storage, such as disk. A disk may be partitioned into tracks, which are partitioned into sectors.
Modern storage subsystem users are demanding advanced functions, such as point-in-time recovery, continuous bidirectional replication, encryption, etc. Older storage subsystems have supported data compression, a feature liked by users, but not supported in some recent storage subsystems. Also, storage subsystems are increasingly being built on top of general purpose computers, hence scalability, reliability, and recoverability for hardware and software failures are also needed. It takes significant time to develop and integrate new features in storage subsystems.
A storage controller is part of a storage subsystem and includes a storage virtualization engine that take sectors from a plurality of physical disks and makes them available to one or more host processors at host computers as sectors of a plurality of “virtual” disks, which is analogous to logical memory for multiprocessing host computers. The storage virtualization engine maintains and manages the mapping of sectors from physical to logical disks. Input/Output (I/O) commands are made to logical disk sectors and sent to the storage virtualization engine. The storage virtualization engine redirects the I/O commands to the appropriate sector on a physical disk. In some implementations, a hardware cache, managed by the storage virtualization engine, is a front end for the physical disks, and I/O commands may be serviced from the cache. Storage virtualization engine are described in IBM Corp. Application Development Guide: Programming Client Applications. IBM Redbooks, 2002, which is incorporated by reference herein in its entirety.
A Relational DataBase Management System (RDBMS) uses relational techniques for storing and retrieving data in a relational database. Relational databases are computerized information storage and retrieval systems that are organized into tables that consist of records and columns of data. The records may be called tuples or records. A database typically has many tables, and each table typically has multiple records and multiple columns. A client computer typically issues requests to a server that includes a RDBMS. The RDBMS provides the client computer access to storage (e.g., a relational database).
RDBMS software may use a Structured Query Language (SQL) interface. The SQL interface has evolved into a standard language for RDBMS software and has been adopted as such by both the American National Standards Institute (ANSI) and the International Standards Organization (ISO).
In some systems, a relational database engine consists of the following software components: (i) a network communications manager component, (ii) a query management component, (iii) a record management component, and (iv) a page buffer management component.
In conventional systems, a source of performance overhead is found in client computer-server processing, which is handled by the network communications manager component and the query management component. In particular, the client computers and servers may reside on a multiplicity of operating systems and/or processor platforms. Representation of basic data types (e.g., integers) may be different on different platforms. An RDBMS may be capable of concurrently servicing requests from client computers running on various platforms. Hence, the network communications manager component has a rich set of translation capabilities to mask the impact of the variations between platforms on the result of database operations conducted by the client computer or server.
Queries (e.g., in the form of SQL statements) are processed by the query management component. The queries may be parsed and tokenized before being sent to the RDBMS. The tokens are retrieved and transformed into data structure elements that are used to drive query processing. When an SQL query arrives at an RDBMS server, the SQL query is analyzed, and the most optimum way to execute the SQL query is determined. Then, run time structures needed to drive the RDBMS record management component are constructed. The record management component is driven with these initialized data structures. Query analysis, query optimization, and setting up of the run time data structures consume CPU overhead.
U.S. patent application entitled “Apparatus, System, and Method for Supporting Storage Functions Using an Embedded Database Management System”, having application Ser. No. 10/958,954, and filed on Oct. 5, 2004 (referred to hereinafter as the '954 application), is incorporated by reference herein in its entirety. The '954 application described a table module with a first field containing a storage identifier and a second field having data contents. The table module is configured to emulate a virtual storage device and maintain a plurality of records.
The data present on logical disks (i.e., identified by Logical Unit Numbers (LUNs)), as well as on the physical disks, may be represented as sector vectors. A sector vector may be described as a set of consecutive sectors. A sector vector may be stored in a table. In addition, compression and encryption render fixed length sectors into variable sized compressed or encrypted sectors. When sectors are stored in records of tables, compression and encryption result in variable length records stored in the tables. Two techniques for tackling the problems of variable length records include a log structure array approach and a DBMS-based approach. The log structure array approach appends every updated or newly created sector to the end of the sector vector, while simultaneously reclaiming space left by holes (M. Rosenblum and J. K. Ousterhout, “The Design and Implementation of a Log-Structured File System”, ACM Transactions on Computer Systems, V 10, No. 1, pages 26-52, 1992, which is incorporated by reference herein in its entirety). The DBMS-based approach introduces some extra space for a set of variable length sectors to accommodate changes in the size of a sector. This maintains the sequentiality of records better than a log structured array. This feature is important for commercial and scientific applications that are known to have highly sequential I/O reference patterns (International Business Machines Corporation, MVS/DFP V3R3 System Programming Reference, IBM Redbooks, 1996; M. Poess and D. Potapov, Data Compression in Oracle, In International Conference on Very Large Data Bases, pages 937-947, 2003; which are incorporated by reference herein in their entirety), and the expectation of these applications are better preserved by maintaining sequentiality.
Since the compressed sectors have variable length, and the location of the sectors may vary over time, indirection may be used to locate the compressed sectors. Therefore, an index is built on sector numbers (i.e., that correspond to logical sector numbers of sectors in logical storage) using intrapage indirection (i.e., the forward address in the data page serves as the technique for access). The forward address allows records to be moved around in pages without changing the index. The execution of a SQL operation, therefore, picks the access path through the index to access the data. An index is usually structured as a B-tree, and CPU overhead of navigating a B-tree may be large. This happens when the I/O size is not large, and therefore the B-tree traversal cost is much larger than index page access through the leaf page links; or when the size of the virtual sector vector changes, referred to as LUN resizing. The change in LUN size may be significant. The change in LUN size may result in a large amount of insertion and deletion of table records. Thus, extensive amount of B-tree non-leaf node splits and merges may be observed during the LUN resizing. From the above concerns, reducing index operation cost becomes important.
Thus, there is a need in the art for implementing storage management functions in a data store system, such as an RDBMS.